Image segmentation and edge detection for multispectral (MS) and hyperspectral (HS) images can be an inherently difficult problem since gray-scale images associated with individual spectral bands may reveal different edges. Segmentation algorithms for gray-scale images utilize basic properties of intensity values such as discontinuity and similarity. Popular gray-scale edge detectors include Canny, Sobel, and Prewitt detectors, to name just a few. The transition from a gray-scale to a multicolor image complicates edge detection significantly: the standard definition of a gray-scale edge as a “ramp” or “ridge” between two regions is no longer appropriate because a multicolor image has multiple image planes (channels) corresponding to different spectral bands. Moreover, depending on the composition of the scene, two distinct spectral (color) regions may exhibit the same intensity for one or more bands and, in this case, the edge between the two regions is termed isoluminant. An isoluminant edge is therefore characterized by a jump in color rather than a jump in intensity. As a result, a standard gradient-based operator cannot detect isoluminant edges easily because they usually do not exhibit an intensity ramp that can be estimated by the magnitude of such an operator.
The extension of gray-scale edge detection to multicolor images has followed three principal paths. A straightforward approach is to apply differential operators, such as the gradient, separately to each image plane (e.g., to each color slice of the three dimensional MS or HS image) and then consolidate the information to obtain edge information. Several key drawbacks of such a straightforward approach have been identified. First, while the combinations of different image planes can generally define edges, these edges may be missing in some of the image planes. Second, processing image planes separately disregards potential correlation across image planes. Third, integration of information from separate image planes is not trivial and is often done in an ad hoc manner. Moreover, in cases when an edge appears only in a subset of image planes, there are no standard, principle-based approaches to fuse the information from different planes. In recent years, new gray-scale algorithms were presented in the community but they all suffer from the scalability problem when applied to multicolor images.
A second approach for multicolor edge detection is to embed the variations of all color channels in a single measure, which is then used to obtain the edge maps. Typically, this approach is developed by starting from a given gray-scale operator, which is then consistently extended to multicolor images. Two representative examples of this approach are the multicolor gradient (MCG), and the morphological color gradient (MoCG). The MCG operator represents a consistent extension of the standard gradient operator to multicolor images and it measures the local steepness of the multicolor image considered as a manifold embedded in the Euclidean space. A hyper-pixel belongs to a multicolor edge if the local steepness of the manifold, as measured by MCG, exceeds a given threshold. Similarly, the MoCG operator is a consistent extension of the morphological gray-scale gradient operator to multicolor images. Such an operator is defined as the difference of the dilation and the erosion operators applied to a given structuring element. Because the MCG and the MoCG edge detectors simultaneously utilize spatial and spectral information, they are examples of joint spatio-spectral image-processing algorithms. The MCG algorithm and its related algorithms have been used with great success in digital image processing. However, for spectral images with a large number of bands, the number of mathematical operations required by the MCG algorithm can be prohibitively high, making the MCG algorithm not suitable for some big-data applications.
Another approach that falls into the category of joint spatio-spectral algorithms is the order-statistics approach and its extensions. In general, these algorithms consider the data as a discrete vector field and they utilize an R-ordering method to define a color edge detector using the magnitudes of linear combinations of the sorted vectors. Another algorithm in this category performs a mapping of the color image into a feature space, which considers features such as the local contrast, the edge connectivity, the color contrast similarity, and the orientation consistence. These features are merged together to create a single feature that is compared to a threshold to generate the final edge map. Similar approaches and more sophisticated estimators of the color gradient can be found in the literature on such processing.
A third approach for multicolor edge detection is to aim the algorithm to detect solely the changes between the materials present in the imaged scene. For example, the Hyperspectral/Spatial Detection of Edges (HySPADE) algorithm transforms the data cube into a spectral angle (SA) cube by calculating the SA between each hyperpixel in the cube with every other pixel. As a result, the third dimension in the original cube is replaced by the SA results, where jumps in the SA represent changes in the materials. The positions of these jumps are mapped back in to the original data cube, and the final edge map is derived upon statistical accumulation of edge information contained in every SA-cube. One important distinction between the MCG algorithm and the HySPADE algorithm is that the former utilizes both spectral and spatial information to detect the edges, while the HySPADE utilizes solely spectral information to unveil the boundaries of the material composition.